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Assessing  Prior  Experience  in  the  Selection 
of  Air  Traffic  Control  Specialists 


INTRODUCTION 

The  U.S.  Office  of  Personnel  Management  (OPM) 
supports  the  federal  government  in  workforce  hiring.  One 
aspect  of  their  work  is  to  publish  qualification  standards  for 
occupations  within  the  federal  government.  Qualification 
standards,  developed  in  consultation  with  the  appropriate 
federal  agency  (OPM,  2009),  detail  the  educational  and 
experience  requirements  applicants  should  have  to  qualify 
for  specific  positions  within  the  federal  government. 

Qualification  standards  may  represent  an  occupational 
group  or  series.  One  such  occupation  is  the  Air  Traffic 
Control  (ATC)  Series  2152.  For  position  classification  pur¬ 
poses,  the  ATC  Series  is  part  of  the  Transportation  Group, 
2100-2199.  There  is  no  group  qualification  standard  for 
the  Transportation  Group,  but  individual  occupational 
requirements  do  exist.  For  the  ATC  Series  2152,  appli¬ 
cants  may  meet  minimum  qualification  requirements  on 
the  basis  of  education  or  experience.  To  qualify  based  on 
education,  applicants  must  have  earned  at  least  a  bachelor  s 
degree.  Experience  is  divided  into  general  and  specialized 
experience.  General  experience  is  used  to  hire  applicants 
with  no  background  in  ATC.  However,  applicants  must 
have  progressively  responsible  work  experience  that  dem¬ 
onstrates  the  potential  for  learning  and  performing  ATC 
work  (Air  Traffic  Controller  Occupational  Handbook, 
2003).  Specialized  experience  is  experience  as  an  air  traffic 
controller  in  a  civilian  or  military  ATC  facility. 

In  addition  to  education  and  experience,  the  ATC 
Series  2152  qualification  standard  includes  a  list  of  alter¬ 
nate  requirements  to  qualify  applicants  for  selection  as  an 
ATCS.  The  alternate  requirements  include  the  specialized 
experience  requirement  and  extend  that  requirement  to 
include  other  examples  of  prior,  relevant  experience.  The 
alternate  requirements  also  include  the  general  experience 
requirement,  accompanied  by  an  additional  written  test 
requirement.  Alternate  requirements  can  apply  to  either 
civilian  or  military  hires.  Applicants  hired  with  prior, 
relevant  experience,  as  provided  under  the  specialized 
experience  provision  of  the  qualification  standard  or  the 
alternate  requirements,  may  be  exempted  from  attending 
the  ATC  basic  training  course  for  new  hires  at  the  FAA 
Academy  but  generally  must  attend  and  successfully 
complete  the  initial  training  course  that  follows  the  basic 
training  course  before  reporting  to  a  facility. 

To  be  useful  in  selecting  ATCSs  from  civilian  or  military 
applicants,  the  OPM  standard  and  alternate  requirements 
should  reflect  current  military  jobs  or  work  experience, 


as  well  as  be  able  to  discriminate  among  the  applicants 
by  distinguishing  the  effective  from  the  ineffective  per¬ 
formers.  The  OPM  standard  and  alternate  requirements 
should  also  reflect  current  FAA  personnel  practices.  The 
purpose  of  the  present  study  was  to  assess  the  OPM 
qualification  standard  for  the  ATC  Series  2152,  focusing 
on  the  specialized  experience  or  alternate  requirements 
used  to  qualify  and  select  ATCSs  from  among  applicants 
with  relevant  experience.  These  include  applicants  with 
civilian  or  military  ATC  or  aviation-related  experience. 
An  outcome  of  the  research  will  be  to  provide  the  FAAs 
Office  of  Human  Resource  Management  (AHR)  with 
recommendations  for  revising  the  alternate  requirements 
based  on  both  current  military  practices  and  the  utility 
of  the  alternate  requirements  for  selecting  ATCSs  from 
among  those  applicants  with  relevant  experience. 

The  FAA  ATCS  Applicant  Qualification  Process 

Currently,  applicants  seeking  employment  as  ATCSs 
begin  the  process  by  applying  on-line  in  response  to  a 
specific  vacancy  announcement.  Vacancy  announcements 
for  ATCSs  target  three  major  populations  of  applicants. 
One  group,  referred  to  as  general  public  applicants,  has  no 
prior  experience  in  ATC.  The  second  group  has  experience 
in  ATC  and,  depending  on  their  experience,  may  qualify 
to  apply  under  vacancy  announcements  for  (a)  Veterans 
Recruitment  Appointment  (VRA)  authority,  (b)  former 
Department  of  Defense  air  traffic  controllers  eligible  for 
transfer  or  reinstatement  or  retired  military  controllers, 
(c)  former  FAA  ATCSs  who  are  reinstatement  eligible, 
or  (d)  holders  of  a  control  tower  operator’s  license  with  a 
facility  rating  of  tower/cab.  The  third  group  of  applicants 
targeted  for  hiring  by  the  FAA  is  graduates  of  an  Air 
Traffic-Collegiate  Training  Initiative  (AT-CTI)  program 
from  an  accredited  college  or  university. 

Once  an  application  is  made  in  response  to  a  specific 
vacancy  announcement,  the  applicants  qualifications  are 
screened  against  the  criteria  for  that  announcement,  as 
well  as  for  more  general  criteria  required  of  all  applicants. 
For  example,  to  qualify,  all  applicants  must  be  citizens  of 
the  United  States.  Applicants  are  judged  to  be  qualified 
if  they  meet  both  the  qualification  criteria  for  the  specific 
vacancy  announcement  and  the  more  general  criteria  for 
all  ATCSs.  As  part  of  the  minimum  qualifications,  those 
applicants  with  no  experience  as  ATCSs  (i.e.,  general 
public)  and  those  who  apply  as  AT-CTI  graduates  must 
take  and  pass  the  Air  Traffic- Selection  and  Training  (AT- 
SAT)  test  battery.  Applicants  with  ATCS  experience  do 
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Figure  1 .  ATCS  Hiring  and  Academy  Training  Timeline 


not  take  AT- SAT.  Figure  1  shows  the  ATCS  hiring  and 
Academy  training  timeline. 

The  OPM  qualification  standard  may  not  be  as  detailed 
as  the  specific  vacancy  announcements  but  should  be 
consistent  with  the  FAA  ATCS  qualification  process  to 
provide  accurate  information  to  ATCS  applicants.  The 
education  and  experience  requirements  listed  in  the  OPM 
qualification  standard  for  the  ATCS  are  consistent  with 
FAA  qualification  processes,  with  the  exception  of  pay  scale 
information  related  to  the  General  Schedule  system,  which 
is  no  longer  used  by  the  FAA.  In  addition  to  education  and 


experience  requirements,  the  current  OPM  qualification 
standard  defines  seven  alternate  requirements  as  qualifying 
experience.  These  alternate  requirements,  listed  below  in 
Table  1,  are  the  focus  of  this  investigation. 

Two  studies  were  conducted  to  address  these  alternate 
requirements.  Where  data  were  available,  a  quantitative 
analysis  was  conducted  to  assess  the  degree  to  which  the 
alternate  requirement  was  related  to  training  performance 
in  the  FAA  Academy.  This  analysis  is  presented  in  Study  1 . 
A  second  study,  a  qualitative  analysis  based  on  interview 
data  and  a  review  of  pertinent  documents,  was  conducted 


Table  1 .  Alternate  Requirements  for  Qualifying  ATCS  Applicants 


1. 

Hold  or  have  held  an  appropriate  facility  rating  and  have  actively  controlled  air  traffic  in 
civilian  or  military  ATC  terminals  or  centers; 

2. 

Hold  or  have  held  an  FAA  certificate  as  a  dispatcher  for  an  air  carrier; 

3. 

Hold  or  have  held  an  instrument  flight  rating; 

4. 

Hold  or  have  held  an  FAA  certificate  as  a  navigator  or  have  been  fully  qualified  as  a 
navigator/bombardier  in  the  Armed  Forces; 

5. 

Have  350  hours  of  flight  time  as  a  copilot  or  higher  and  hold  or  have  held  a  private  certificate  or 
equivalent  Armed  Forces  rating; 

6. 

Have  served  as  a  rated  Aerospace  Defense  Command  Intercept  Director; 

7. 

Meet  the  requirements  for  GS-5  and  pass  the  written  test  with  an  appropriately  higher  score. 
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to  assess  those  alternate  requirements  for  which  no  data 
were  available.  Each  of  the  requirements  is  addressed  with 
recommendations  for  retention,  revision  and  retention, 
or  elimination. 

STUDY 1 
METHOD 

This  study  uses  a  quantitative  approach  to  address 
alternate  requirements  1-5  (see  Table  1). 

Participants 

Biographical  information,  pre-employment  screening 
test  scores,  and  performance  scores  in  the  FAA  Academy, 
gathered  from  12,858  prospective  ATCSs  attending  the 
FAA  Academy  between  1986  and  1 992,  were  used  in  Study 
1 .  The  large  amount  of  data  available  during  this  timeframe 
was  due  to  the  August  1981  strike  of  the  Professional  Air 
Traffic  Control  Organization  (PATCO),  the  subsequent 
firing  of  more  than  10,000  ATCSs  (Schroeder,  Broach,  & 
Farmer,  1998),  and  the  FAA  hiring  a  considerable  number 
of  ATCSs  to  replace  the  fired  controllers.  Hiring  during  the 
mid-and  late  90s  and  into  the  early  2 1  st  century  occurred 
at  a  much  slower  pace  than  during  the  decade  after  the 
strike.  The  workforce  stabilized  after  1992  and  the  FAA 
hired  only  about  100  controllers  per  year  through  1997 
(Aul,  1998).  More  recently,  from  2000-20 1 0,  biographical 
data  on  newly  hired  ATCSs,  which  might  have  been  used 
for  this  study,  were  not  gathered  for  those  hired  based  on 
prior  experience.  Data  were  only  gathered  on  prospective 
ATCSs  hired  from  the  general  public,  most  of  whom 
would  have  had  little  prior  experience. 

Besides  being  the  best  available  data  for  use  in  this 
study,  there  is  also  some  evidence  to  suggest  that  prospec¬ 
tive  ATCSs  hired  between  1986  and  1992  are  similar  in 
some  respects  to  those  hired  more  recently.  For  example,  all 
ATCS  applicants  must  be  U.S.  citizens.  The  vast  majority 
of  applicants  then  and  now  must  be  initially  appointed 
before  reaching  the  age  of  31,  with  the  only  exception 
being  retired  military  controllers.  In  addition,  Cannon 
and  Broach  (2011)  found  that  controllers  hired  in  2009 
responded  much  like  controllers  hired  between  1986  and 
1992  regarding  factors  that  influenced  their  choice  of  the 
controller  occupation.  In  follow-on  research,  Cannon  and 
Broach  (personal  communication,  December  19,  2011) 
compared  the  extent  to  which  controllers  hired  between 
1986  and  1992  reported  having  had  similar  prior  experi¬ 
ences  as  those  hired  in  2009.  Preliminary  results  indicate 
that  the  numbers  of  new  hires  with  experience  in  the  areas 
investigated  in  the  current  study  are  similar,  except  for 
experience  as  a  private  and/or  commercial  pilot.  More  of 
the  2009  hires  reported  having  experience  as  a  private  and / 
or  commercial  pilot  than  those  hired  from  1986-1992. 


Regarding  the  data  gathered  between  1986  and  1992, 
16%  reported  having  prior  experience  in  ATC  or  as  a 
pilot,  and  approximately  73%  provided  information  on 
questions  of  interest  to  this  study.  The  exact  number  of 
responses  varied  by  the  question  selected.  The  number  of 
respondents  per  question  ranged  from  9,333  to  9,354. 
Those  responding  to  the  questions  of  interest  were 
similar  in  characteristics  to  the  larger  sample.  Therefore, 
demographic  information  is  only  reported  for  the  larger 
sample.  The  majority  of  those  in  the  sample  were  male 
(80%)  and  white  (89%).  Their  average  age  at  entry  into 
the  FAA  Academy  was  25.81  (. sd  =  2.89).  Most  had  some 
college  experience  (89%),  with  the  rest  having  graduated 
from  high  school  (1 1%). 

Measures 

Biographical  Questionnaire.  The  FAA  routinely  uses 
a  biographical  questionnaire  (BQ)  to  collect  biographical 
information  from  prospective  ATCSs.  This  information  is 
usually  collected  after  the  prospective  ATCSs  are  selected 
and  report  to  training.  Items  on  the  original  version  of  the 
BQ(VanDeventer,  Taylor,  Collins,  &  Boone,  1983)  were 
primarily  designed  to  gather  information  on  education 
and  prior  experience  of  the  prospective  ATCSs.  The  BQ 
has  been  revised  multiple  times  for  research  purposes  and 
to  support  human  resources  initiatives  in  recruitment, 
selection,  and  training  (Farmer,  2002).  Farmer  (2002) 
provides  an  overview  of  the  development  and  use  of  the 
BQto  predict  performance  and  attrition  of  ATCSs. 

We  identified  items  on  the  BQthat  were  judged  to  be 
related,  either  directly  or  indirectly  to  Alternate  Require¬ 
ments  1-5  (see  Table  1).  There  were  no  BQ  items  found 
to  be  related  to  Alternate  Requirements  6  or  7,  which 
will  be  addressed  in  Study  2  of  this  report.  Responses  to 
all  relevant  BQ  items  were  grouped  into  two  categories, 
indicating  that  the  respondent  either  did  or  did  not  have 
the  relevant  experience.  Responses  to  these  items  were 
then  analyzed  to  determine  if  there  was  a  relationship 
between  the  types  of  experience  identified  in  the  alternate 
requirements  and  FAA  Academy  training  performance. 

FAA  Academy  Training  Performance.  The  criterion 
for  our  analysis  was  FAA  Academy  training  performance. 
Prospective  ATCSs  were  grouped  into  two  categories.  They 
either  passed  or  did  not  pass  Academy  training.  Those  who 
did  not  pass  Academy  training  either  withdrew  prior  to 
completion  of  training  or  failed  training. 

Procedures 

Reported  in  Table  2  are  the  BQ  items  selected  for 
analysis  and  the  number  of  participants  responding  to 
each  BQ  item,  along  with  the  relevant  alternate  require¬ 
ment.  For  the  first  alternate  requirement,  three  BQ  items 
judged  to  be  related  to  the  requirement  were  selected  for 
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Table  2.  Alternate  Requirements  and  Related  BQ  Items 


Alternate  Requirement 

BQ  Items 

1. 

Hold  or  have  held  an  appropriate  facility 
rating  and  have  actively  controlled  air 
traffic  in  civilian  or  military  ATC 
terminals  or  centers; 

BQ33  -  Do  you  have  a  prior  Control  Tower 

Operator  Rating?  (n=  9,354) 

BQ34  -  Do  you  have  a  prior  Air  Traffic  Control 
Specialist  (ATCS)  rating?  (n=9,333) 

BQ35  -  Do  you  have  prior  Instrument  Flight  Rules 
(IFR)  operations  experience?  (n  =  9,349) 

2. 

Hold  or  have  held  an  FAA  certificate  as  a 
dispatcher  for  an  air  carrier; 

BQ  70  -  Indicate  whether  or  not  you  currently,  or 
have  ever  possessed  any  of  the 
ratings/certificates/licenses:  Dispatch-air  carrier. 
(n=9,339) 

3. 

Hold  or  have  held  an  instrument  flight 
rating; 

BQ  65  -  Indicate  whether  or  not  you  currently,  or 
have  ever  possessed  any  of  the 
ratings/certificates/licenses:  Instrument.  (n=9,334) 

4. 

Hold  or  have  held  an  FAA  certificate  as  a 
navigator  or  have  been  fully  qualified  as  a 
Navigator/Bombardier  in  the  Armed 

Forces; 

BQ  71  -  Indicate  whether  or  not  you  currently,  or 
have  ever,  possessed  any  of  the 
ratings/certificates/licenses:  Navigator/Bombardier. 
(n=9,340) 

5. 

Have  350  hours  of  flight  time  as  a  copilot 
or  higher  and  hold  or  have  held  a  private 
certificate  or  equivalent  Armed  Forces 
rating; 

BQ  56  -  Indicate  whether  or  not  you  currently,  or 
have  ever  possessed  any  of  the 
ratings/certificates/licenses:  Private  Pilot.  (n=9,340) 

BQ  57  -  Indicate  whether  or  not  you  currently,  or 
have  ever  possessed  any  of  the 
ratings/certificates/licenses:  Commercial  Pilot. 
(n=9,339) 

analysis.  For  Alternate  Requirements  2-4,  one  BQ  item 
per  alternate  requirement  was  selected  for  analysis.  There 
were  two  relevant  BQitems  for  Alternate  Requirement  5. 

Analyses 

Independent  chi-square  analyses  (Field,  2009)  were 
performed  to  assess  the  extent  to  which  the  selected  BQ 
items,  operating  as  surrogates  for  the  alternate  require¬ 
ments,  were  related  to  Academy  training.  The  2X2 
contingency  tables  used  in  the  chi-square  analyses  were 
based  on  prospective  ATCSs  indicating  that  they  had  or 
did  not  have  the  relevant  experience  and  passed  or  did  not 
pass  the  FAA  Academy  training.  Not  passing  Academy 
training  included  either  withdrawing  before  the  end  of 
training  or  failing  training.  Both  withdrawing  and  failing 
the  Academy  were  considered  to  be  unsuccessful  training 
outcomes.  A  binary  logistic  regression  was  also  computed 
to  develop  a  model  of  the  BQ  items  best  suited  for  pre¬ 
dicting  Academy  training. 


RESULTS 

The  results  are  presented  by  alternate  requirement.  The 
number  of  BQitems  selected  to  assess  each  alternate  require¬ 
ment  varies  based  on  the  number  of  relevant  items  available 
for  analysis. 

Alternate  Requirement  1 :  Hold  or  have  held  an  appropri¬ 
ate  facility  rating  and  have  actively  controlled  air  traffic  in 
civilian  or  military  ATC  terminals  or  centers. 

There  were  three  items  selected  from  the  BQ  as  relat¬ 
ing  to  Alternate  Requirement  1.  Each  item  reflects  prior 
experience  in  a  civilian  or  military  ATC  facility.  Results  are 
presented  by  BQ  item. 

Do  you  have  a  prior  Control  Tower  Operator  ( CTO)  Rating? 
Of  the  9,354  respondents  to  this  question,  8,408  did  not 
have  a  CTO  rating  and  946  indicated  that  they  had  a  CTO 
rating.  The  number  of  those  who  did  not  have  a  CTO  rat¬ 
ing  and  passed  Academy  training  was  4,4 14  (52.5%),  with 
3,994  (47.5%)  withdrawing  or  failing.  The  number  of  those 
who  had  a  CTO  rating  and  passed  Academy  training  was 
579  (61.2%),  with  3 67  (38.8%)  withdrawing  or  failing  the 
Academy.  The  assumption  for  this  and  all  other  chi-square 
analyses  reported  in  this  study  were  met  in  that  all  frequencies, 
observed  and  expected,  were  greater  than  5  (see  Appendix  A) . 
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Figure  2.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  a  CTO  Rating 


The  chi-square  analysis  indicated  that  there  was 
a  significant  relationship  between  CTO  rating  and 
Academy  training  performance  (%2  =  25.91,  df  =  1,  Sig 
(p=.000)) .  As  shown  in  Figure  2,  respondents  with  a  prior 
CTO  rating  passed  more  often  than  would  be  expected 
(579/505)and  withdrew  or  failed  less  often  than  would 
be  expected  (367/441).  The  relationship  was  reversed  for 


respondents  without  a  prior  CTO  rating.  They  passed 
less  often  than  would  be  expected  (4,414/4,488)  and 
withdrew  or  failed  more  often  than  would  be  expected 
(3,994/3,920). 

Because  these  are  categorical  data,  the  effect  size  can 
be  determined  by  calculating  an  odds  ratio  (Field,  2009). 
The  steps  in  calculating  an  odds  ratio  are  described  below. 


First,  the  odds  that  a  person  with  a  CTO  rating  would  pass  Academy  training  were  determined  by  dividing  the 
number  of  people  with  a  CTO  rating  who  passed  Academy  training  by  the  number  with  a  CTO  rating  who  withdrew 
or  failed.  In  this  case,  579  people  holding  a  CTO  rating  passed  training,  while  367  people  holding  a  CTO  rating  with¬ 
drew  or  failed.  Thus,  the  odds  that  a  person  with  a  CTO  rating  would  pass  Academy  training  were  1.58  to  1  (Eq.  1). 

Number  with  a  CTO  rating  who  passed 

OddS(passing  Acac|emy  Trajnjng  wjth  a  CTO  Rating)  Number  with  a  CTO  rating  who  withdrew  or  failed 

_  579 

"  367 

=  1.58  Eq.  1 

The  next  step  is  to  calculate  the  odds  that  a  person  without  (w/o)  a  CTO  rating  has  of  passing  Academy  training. 
This  is  done  by  dividing  the  number  of  people  who  did  not  have  a  CTO  rating  and  passed  by  the  number  who  did 
not  have  a  CTO  rating  and  withdrew  or  failed  (4414/3994  =  1.11)  .  The  odds  that  a  person  without  a  CTO  rating 
will  pass  Academy  training  are  1.11  (Eq.  2). 

Odds  Number  w/o  a  CTO  rating  who  passed _ 

(Passing  Academy  Training  w/o  a  cto  Rating)  “  Number  w/o  a  CTO  rating  who  withdrew  or  failed 

_  4414 
3994 

=  1.11  Eq.  2 

The  odds  ratio  is  then  calculated  by  dividing  the  odds  of  passing  if  one  has  a  CTO  rating  by  the  odds  of  passing  if 
one  does  not  have  a  CTO  rating  (1.58/1.11  =  1.42;  Eq.  3). 

OddS  Ratio  =  QddS(passing  /\cac|emy  Training  with  a  CTO  Rating) 

OddS(Passing  Academy  Training  w/o  a  CTO  Rating) 

=  1.58 
1.11 

=  1.42  Eq.  3 


5 


Figure  3.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  an  ATCS  Rating 


Figure  4.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  IFR  Experience 


Thus,  the  odds  that  person  with  a  CTO  rating  would 
pass  Academy  training  was  1 .42  times  higher  than  if  they 
had  attended  the  Academy  and  did  not  have  a  CTO 
rating.  This  approach  for  calculating  the  effect  size  is 
used  in  all  subsequent  analyses  in  this  study. 

Do  you  have  a  prior  Air  Traffic  Control  Specialist  (ATCS) 
rating?  Of  the  9,333  respondents  to  this  question,  8,892 
(95.3%)  did  not  have  a  prior  ATCS  rating,  and  441 
(4.7%)  indicated  that  they  had  a  prior  ATCS  rating.  The 
number  of  those  who  did  not  have  a  prior  ATCS  rating 
and  passed  Academy  training  was  4,696  (52.8%),  with 
4,196  (47.2%)  withdrawing  or  failing.  The  number  of 
those  who  passed  Academy  training  with  an  ATC  rating 
was  282  (63.9%),  with  159  (36.1%)  withdrawing  or 
failing  (see  Appendix  B) . 

The  chi-square  analysis  indicated  that  there  was 
a  significant  relationship  between  ATCS  rating  and 
Academy  training  performance  (%2=  20.93,  df  =  1,  Sig 
(p=.000)).  As  shown  in  Figure  3,  respondents  with  a 
prior  ATCS  rating  passed  more  than  would  be  expected 
(282/235.2)  and  withdrew  or  failed  less  than  would  be 
expected  (159/205.8).  The  relationship  was  reversed  for 
respondents  with  no  prior  ATCS  rating.  They  passed  less 


than  would  be  expected  (4,696/4,742.8)  and  withdrew 
or  failed  more  than  would  be  expected  (4,196/4,149.2). 

The  odds  that  a  person  with  a  prior  ATCS  rating 
would  pass  Academy  training  were  1.58  times  higher 
than  if  they  had  attended  the  Academy  and  did  not  have 
a  prior  ATCS  rating. 

Do  you  have  prior  IFR  (Instrument  Flight  Rules) 
Operations  experience ?  Of  the  9,349  respondents  to  this 
question,  7,774  (83.2%)  did  not  have  IFR  operations 
experience  and  1,575  (16.8%)  indicated  that  they  had 
prior  experience  in  IFR  operations.  The  number  of  those 
who  did  not  have  IFR  operations  and  passed  Academy 
training  was  4,031  (51.9%),  with  3,743  (48.1%)  with¬ 
drawing  or  failing.  The  number  of  those  who  had  IFR 
operations  experience  and  passed  Academy  training  was 
960  (61.0%),  with  615  (39%)  withdrawing  or  failing 
(see  Appendix  C). 

The  chi-square  analysis  indicated  that  there  was  a  sig¬ 
nificant  relationship  between  IFR  operations  experience 
and  Academy  training  performance  (%2  =  43.58,  df=  1,  Sig 
(p=.000)).  As  can  be  seen  in  Figure  4,  respondents  with 
experience  in  IFR  operations  passed  Academy  training 
more  than  would  be  expected  (960/840.8)  and  withdrew 
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Figure  5.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  Dispatcher  Experience 


or  failed  less  than  would  be  expected  (615/734.2).  The 
relationship  was  reversed  for  respondents  with  no  experi¬ 
ence  in  IF  R  operations.  They  passed  Academy  training  less 
than  would  be  expected  (4,031/4,150.2)  and  withdrew 
or  failed  more  than  would  be  expected  (3,743/3,623.8). 

The  odds  that  a  person  with  IFR  experience  would 
pass  Academy  training  were  1 .44  times  higher  than  if 
they  had  attended  the  Academy  and  did  not  have  IFR 
experience. 

Alternate  Requirement  2.  Hold  or  have  held  an  FAA 
certificate  as  a  dispatcher  for  an  air  carrier. 

A  single  item  on  the  BQ  was  found  to  reflect  the 
second  alternate  requirement. 

Indicate  whether  or  not  you  currently \  or  have  ever ; 
possessed  any  of  the  ratings/ certificates/ licenses:  Dispatch-air 
carrier.  Of  the  9,339  respondents  to  this  question,  9,288 
(99.5%)  did  not  have  a  rating,  certificate  or  license  as  a 
dispatch-air  carrier  and  5 1  (.5%)  indicated  that  they  cur¬ 
rently  had  or  had  previously  possessed  a  rating,  certificate 
or  license  as  a  dispatcher  for  an  air  carrier.  The  number  of 
those  without  a  dispatch-air  carrier  rating,  certificate  or 
license  and  passed  Academy  training  was  4,958  (53.4%), 
with  4,330  (46.6%)  withdrawing  or  failing.  The  number 
of  those  with  dispatch-air  carrier  experience  who  passed 
Academy  training  was  32  (62.7%),  with  19  (37.3%) 
withdrawing  or  failing  (see  Appendix  D). 

The  chi-square  analysis  did  not  show  a  significant  rela¬ 
tionship  between  having  a  rating,  certificate  or  license  as 
a  dispatch-air  carrier  and  Academy  training  performance 
(%2  =  1.79,  df  =  1,  NS  (p=  .181)).  Although  difficult  to 
see  due  to  the  small  number  of  respondents  with  expe¬ 
rience  as  a  dispatch-air  carrier,  the  pattern  of  responses 


was  similar  to  the  pattern  seen  previously  (see  Figure  5). 
Respondents  with  a  current  or  prior  rating,  certificate  or 
license  as  a  dispatch-air  carrier  passed  slightly  more  often 
than  would  be  expected  (32/27.3)  and  withdrew  or  failed 
slightly  less  often  than  would  be  expected  (19/23.7). 
The  relationship  was  reversed  for  respondents  with  no 
rating,  certificate  or  license  as  a  dispatch-air  carrier.  They 
passed  Academy  training  slightly  less  often  than  would 
be  expected  (4,958/4,962.7)  and  withdrew  or  failed 
Academy  training  slightly  more  often  than  would  be 
expected  (4,330/4,325.3).  However,  as  stated  previously, 
there  was  no  significant  relationship  between  having  a 
rating,  certificate  or  license  as  a  dispatch-air  carrier  and 
Academy  training  performance. 

Because  the  chi-square  analysis  was  not  significant, 
the  odds  ratio  was  not  computed. 

Alternate  Requirement  3.  Hold  or  have  held  an 
Instrument  flight  rating. 

A  single  item  on  the  BQwas  found  to  reflect  Alternate 
Requirement  3. 

Indicate  whether  or  not  you  currently,  or  have  ever  pos¬ 
sessed  any  of  the  ratings/certificates/licenses:  Instrument.  Of 
the  9,334  respondents  to  this  question,  8,919  (95.6%)  did 
not  have  an  Instrument  rating,  and  415  (4.4%)  indicated 
that  they  currently  had  or  previously  had  possessed  an 
Instrument  rating,  certificate  or  license.  The  number  of 
those  who  did  not  have  an  Instrument  rating  and  passed 
Academy  training  was  4,732  (53.1%),  with4, 187  (46.9%) 
withdrawing  or  failing  the  Academy.  The  number  of  those 
who  had  an  Instrument  rating  and  passed  Academy  train¬ 
ing  was  256  (61.7%),  with  159  (38.3%)  withdrawing  or 
failing  (see  Appendix  E). 
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Figure  6.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  an  Instrument  Flight  Rating 


Figure  7.  Observed  and  Expected  AcademyTraining  Performance  With  orWithout  Navigator/Bombardier  Experience 


The  chi-square  analysis  indicated  that  there  was  a 
significant  relationship  between  Instrument  rating  and 
Academy  training  performance  (%2  =  11.87,  df  =  1,  Sig 
(p=.00 1)).  As  can  be  seen  in  Figure  6,  respondents  with 
a  current  or  prior  Instrument  rating  passed  more  than 
would  be  expected  (256/221.8)  and  withdrew  or  failed 
less  than  would  be  expected  (159/193.2).  The  relation¬ 
ship  was  reversed  for  respondents  with  no  current  or  prior 
Instrument  rating.  They  passed  less  often  than  would  be 
expected  (4,732/4,766.2)  and  withdrew  or  failed  more 
often  than  would  be  expected  (4,187/4,152.8). 

The  odds  that  a  person  with  a  current  or  prior  Instru¬ 
ment  rating  would  pass  Academy  training  were  1 .42  times 
higher  than  if  they  had  attended  the  Academy  and  did 
not  have  an  Instrument  flight  rating. 

Alternate  Requirement  4.  Hold  or  have  held  an  FAA 
certificate  as  a  navigator  or  have  been  fully  qualified  as  a 
navigator/bombardier  in  the  Armed  Forces. 

A  single  item  on  the  BQwas  found  to  reflect  the  fourth 
alternate  requirement. 

Indicate  whether  or  not  you  currently,  or  have  ever possessed 
any  of  the  ratings/certificates/licenses:  navigator/bombardier. 
Of  the  9,340  respondents  to  this  question,  9,283  (99.4%) 
did  not  have  a  current  or  prior  rating,  certificate  or  license 


as  a  navigator/bombardier,  while  only  57  (.6%)  indicated 
that  they  currently  had  or  had  ever  had  a  rating,  certifi¬ 
cate  or  license  as  a  navigator/bombardier.  Of  those,  the 
number  who  did  not  have  a  rating,  certificate  or  license 
as  a  navigator/bombardier  and  passed  Academy  training 
was  4,955  (53.4%),  with  4,328  (43.6%)  withdrawing  or 
failing.  The  number  of  those  who  had  a  current  or  prior 
rating,  certificate  or  license  as  a  navigator/bombardier 
and  passed  Academy  training  was  32  (56.1%),  with  25 
(43.9%)  withdrawing  or  failing  (see  Appendix  F). 

The  chi-square  analysis  did  not  show  a  significant  rela¬ 
tionship  between  having  a  current  or  prior  rating,  certificate 
or  license  as  a  navigator/bombardier  and  Academy  training 
performance  (%2  =  .174,  df  =  1,  NS  (p=  .677)).  Although 
difficult  to  see  due  to  the  small  number  of  respondents 
with  experience  as  a  navigator/bombardier,  the  pattern  of 
responses  was  similar  to  the  pattern  seen  previously  (see 
Figure  7).  Respondents  with  a  rating,  certificate  or  license 
as  a  navigator/bombardier  passed  slightly  more  often 
than  would  be  expected  (32/30.4)  and  withdrew  or  failed 
slightly  less  often  than  would  be  expected  (25/26.6).  The 
relationship  was  reversed  for  respondents  without  a  rating, 
certificate  or  license  as  a  navigator/bombardier.  They  passed 
slightly  less  often  than  would  be  expected  (4955/4956.6) 
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Figure  8.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  a  Private  Pilot’s  Rating, 
Certificate  or  License 


and  withdrew  or  failed  slightly  more  often  than  would  be 
expected  (4,328/4,326.4).  However,  as  stated  previously, 
there  was  no  significant  relationship  between  having  a 
rating,  certificate  or  license  as  navigator/bombardier  and 
Academy  training  performance. 

Because  the  chi-square  analysis  was  not  significant, 
the  odds  ratio  was  not  computed. 

Alternate  Requirement  5.  Have  350  hours  of  flight 
time  as  a  copilot  or  higher  and  hold  or  have  held  a  private 
certificate  or  equivalent  Armed  Forces  rating. 

Several  items  on  the  BQ  could  be  used  to  assess  Alter¬ 
nate  Requirement  5.  The  BQ  items  of  relevance  asked  if 
the  prospective  ATCS  had  a  rating,  certificate  or  license  as 
a  pilot  for  multiple  types  of  platforms.  Although  number 
of  hours  flown  was  not  addressed  in  the  BQitem,  for  our 
purposes,  we  selected  two  items.  The  first  addressed  having 
a  private  pilot’s  rating,  certificate  or  license.  The  second 
addressed  having  a  commercial  pilot’s  rating,  certificate  or 
license.  The  minimum  flight  time  requirements  for  pilots 
are  defined  under  Part  61 ,  Title  14  of  the  Code  of  Federal 
Regulations  (Electronic  Code  of  Federal  Regulations, 
2011).  For  a  private  pilot,  the  minimum  requirement  is 
40  hours  of  flight  time;  and  for  a  commercial  pilot,  the 
minimum  requirement  is  250  hours. 

Indicate  whether  or  not  you  currently,  or  have  ever  pos¬ 
sessed  any  of  the  ratings/certificates/licenses:  Private  Pilot.  Of 
the  9,340  respondents  to  this  question,  8,196  (87.8%) 
did  not  have  a  private  pilots  license  and  1,144  (12.2%) 
indicated  that  they  currently  or  had  previously  possessed  a 
rating,  certificate  or  license  as  a  private  pilot.  The  number  of 
those  who  did  not  have  a  private  pilot’s  license  and  passed 
Academy  training  was  4,307  (52.6%),  with  3,889  (47.4%) 


withdrawing  or  failing.  The  number  of  those  who  had  a 
private  pilots  license  and  passed  Academy  training  was 
682  (59.6%),  with  462  (40.4%)  withdrawing  or  failing 
(see  Appendix  G). 

The  chi-square  analysis  indicated  that  there  was  a  signifi¬ 
cant  relationship  between  a  private  pilot’s  rating,  certificate 
or  license  and  Academy  training  performance  (%2  =  20.14, 
df  =  1,  Sig  (p=.000)).  Figure  8  shows  that  respondents 
with  a  private  pilot’s  rating,  certificate  or  license  passed 
more  than  would  be  expected  (682/61 1.1)  and  withdrew 
or  failed  less  than  would  be  expected  (462/532.9).  The 
relationship  was  reversed  for  respondents  without  a  private 
pilot’s  rating,  certificate  or  license.  They  passed  Academy 
training  less  than  would  be  expected  (4,307/4,377.9) 
and  withdrew  or  failed  more  than  would  be  expected 
(3,889/3,818.1). 

The  odds  that  a  person  with  a  private  pilot’s  rating, 
certificate  or  license  would  pass  Academy  training  were 
1 .33  times  higher  than  if  they  did  not  have  a  private  pilot’s 
rating,  certificate  or  license. 

Indicate  whether  or  not  you  currently  or  have  ever possessed 
any  of  the  ratings/certificates/licenses:  Commercial  Pilot.  Of 
the  9,339  respondents  to  this  question,  9,009  (96.5%) 
did  not  have  a  commercial  pilot’s  license  and  330  (3.5%) 
indicated  that  they  currently  or  had  previously  possessed 
a  rating,  certificate  or  license  as  a  commercial  pilot.  The 
number  of  those  who  did  not  have  a  commercial  license 
and  passed  Academy  training  was  4,787  (53.1%),  with 
4,222  (46.9%)  withdrawing  or  failing.  The  number  of 
those  who  had  a  commercial  pilot’s  license  and  passed 
Academy  training  was  201  (60.9%),  with  129  (39.1%) 
withdrawing  or  failing  the  Academy  (see  Appendix  H). 
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Figure  9.  Observed  and  Expected  Academy  Training  Performance  With  or  Without  a  Commercial  Pilot’s  Rating, 
Certificate  or  License 


The  chi-square  analysis  indicated  that  there  was  a 
significant  relationship  between  having  a  commercial 
pilots  rating,  certificate  or  license  and  Academy  training 
performance  (%2=  7.73,  df  =  1,  Sig  (p=.005)).  As  can  be 
seen  in  Figure  9,  respondents  with  a  commercial  pilots 
rating,  certificate  or  license  passed  Academy  training  more 
than  would  be  expected  (201/176.3)  and  withdrew  or  failed 
less  than  would  be  expected  (129/153.7).  The  relationship 
was  reversed  for  respondents  without  a  commercial  pilot’s 
rating,  certificate  or  license.  They  passed  Academy  training 
less  than  would  be  expected  (4,787/4,81 1.7)  and  withdrew 
or  failed  more  than  would  be  expected  (4,222/4,197.3). 

The  odds  that  a  person  with  a  commercial  pilot’s  rat¬ 
ing,  certificate  or  license  would  pass  Academy  training 
were  1.38  times  higher  than  if  they  had  attended  the 
Academy  and  did  not  have  a  commercial  pilot’s  rating, 
certificate  or  license. 

Logistic  Regression 

A  logistic  regression  analysis  was  run  to  determine 
the  relative  weight  of  each  of  the  BQ  items  in  predicting 
Academy  training  performance.  The  eight  items  selected 
from  the  BQas  representing  OPM  alternate  requirements 
for  ATCS  selection  were  entered  into  a  logistic  regression 
model  using  a  forward  stepwise  (likelihood  ratio)  strategy. 
This  method  of  entering  the  items  into  the  model  was 
selected  because  no  theory  exists  regarding  the  relative 
merits  of  the  experiences  represented  in  the  BQ  items 
for  predicting  Academy  training  performance.  Rather, 
our  aim  was  to  develop  a  prediction  model  using  the  best 
predictors  from  among  the  selected  BQ  items. 

The  same  dataset  used  in  the  chi-square  analyses  was 
used  in  the  logistic  regression.  There  were  12,858  cases 
in  the  dataset,  9,273  (72.1%)  included  in  the  analysis. 
The  initial  model  in  the  forward  stepwise  logistic  regres¬ 
sion  was  derived  using  only  the  constant  in  the  regression 
equation.  The  initial  model  maximizes  the  prediction  of 
Academy  training  performance  based  on  the  observed 
data.  In  our  dataset,  4,956  prospective  ATCSs  passed  and 


4,317  withdrew  or  failed  Academy  training.  Since  more 
prospective  ATCSs  passed  than  withdrew  or  failed,  the 
initial  model  predicted  that  all  prospective  ATCSs  would 
pass  Academy  training.  This  model  would  correctly  clas¬ 
sify  Academy  training  performance  53%  of  the  time.  At 
this  stage  in  our  analysis,  none  of  the  BQ  items  had  been 
entered  into  the  model.  However,  the  residual  chi-square 
statistic  for  the  BQ  items  was  68.07  (p  <  .000),  indicating 
that  one  or  more  of  the  BQ  items,  if  added  to  the  baseline 
model,  would  significantly  affect  the  predictive  power  of 
the  model  (see  Table  3). 

The  Rao’s  efficient  score  statistic  (Score)  for  each  of  the 
BQitems  is  listed  in  Table  3.  This  statistic  is  used  by  SPSS 
to  examine  the  significance  of  the  potential  predictors  not 
in  the  model  and  to  select  the  item  that  will  be  entered 
first  into  the  model.  As  can  be  seen,  all  types  of  experience 
except  experience  as  a  dispatcher  for  an  air  carrier  or  as 
a  navigator/bombardier  had  a  significant  Rao’s  efficient 
score  statistic  and  could  potentially  make  a  contribution 
to  the  model.  Having  IFR  operations  experience  had  the 
highest  Rao’s  efficient  score  statistic  and  was  entered  first 
into  the  model.  Other  items  for  inclusion  in  the  model 
were  selected  based  on  their  relative  contribution  given  the 
other  variables  still  not  in  the  model  and  the  contribution 
they  could  make,  given  the  variables  that  had  already  been 
entered  into  the  model. 

Running  the  logistic  regression  using  a  forward 
stepwise  (likelihood  ratio)  entry  method  resulted  in  a 
model  with  three  variables.  Shown  in  Table  4  is  the  log- 
likelihood  statistic  for  each  iteration  of  the  model.  The 
log-likelihood  statistic  of  the  model  with  only  the  constant 
was  12,81 1.039.  As  expected,  the  log-likelihood  statistic 
decreased  with  each  iteration  of  the  model  as  the  unex¬ 
plained  variance  in  the  model  decreased.  However,  the  final 
log-likelihood  statistic  was  still  fairly  large,  indicating  that 
even  after  fitting  the  best  model  possible  from  the  data, 
there  was  a  great  deal  of  unexplained  variance.  This  can 
also  be  seen  by  the  relatively  small  amount  the  predictor 
variables  contributed  to  the  prediction  of  Academy  training 
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Table  3.  Step  0  -  Variables  Not  in  the  Equation 


Experience 

Score 

Degrees  of 
Freedom 

Significance 

Dispatcher- Air  Carrier 

1.294 

i 

.255 

IFR  Operations 

42.383 

i 

.000 

N  avigator/B  omb  ardier 

.310 

i 

.578 

Private  Pilot’s  License 

20.996 

i 

.000 

Commercial  Pilot’s  License 

7.218 

i 

.007 

Instrument  Rating 

12.171 

i 

.000 

CTO  Rating 

24.358 

i 

.000 

ATCS 

22.427 

i 

.000 

Overall  Statistics 

68.071 

8 

.000 

Table  4.  Model  Summary 


Step 

-2  Log  likelihood 

Cox  &  Snell 

R  Square 

Nagelkerke 

R  Square 

1  IFR  Operations 

12,768.276a 

.005 

.006 

2  Private  Pilot’s  License 

12,754.017a 

.006 

.008 

3  ATCS  Experience 

12,746.806* 

.007 

.009 

Table  5.  Variables  in  the  Equation 


B 

S.E. 

Wald 

Degrees  of 
Freedom 

Significance 

Exp(B) 

Step  1 

IFR  Operations 

.370 

.057 

42.097 

i 

.000 

1.447 

Constant 

.078 

.023 

11.634 

i 

.001 

1.081 

Step  2 

IFR  Operations 

.342 

.057 

35.384 

i 

.000 

1.408 

Private  Pilot’s 

.246 

.065 

14.123 

i 

.000 

1.279 

License 

.053 

.024 

4.970 

i 

.026 

1.054 

Constant 

Step  3 

.274 

.063 

19.164 

i 

.000 

1.315 

IFR  Operations 

.260 

.066 

15.622 

i 

.000 

1.296 

Private  Pilot’s 

.298 

.112 

7.100 

i 

.008 

1.347 

License 

.049 

.024 

4.222 

i 

.040 

1.050 

ATCS 

Constant 

performance.  Even  the  best  model  improved  prediction 
over  the  constant  by  only  .007  using  the  Cox  &  Snell 
R  Square  and  .009,  based  on  the  Nagelkerke  R  Square. 

The  three  significant  predictors  were  IFR  Operations 
experience,  having  a  private  pilot’s  license,  and  experience  as 
an  ATCS  (see  Table  5).  The  Wald  statistics,  which  measures 
the  extent  to  which  the  b  coefficients  were  significantly 
different  from  0,  were  significant  for  all  three  predic¬ 
tors,  so  we  assume  that  all  three  are  making  a  significant 
contribution  to  the  prediction  of  who  will  pass  Academy 
training  (Table  5).  Also  shown  in  Table  5  is  the  odds 
ratio  (Exp(B))  for  each  variable.  All  values  were  greater 
than  1 ,  indicating  that  as  the  likelihood  increased  of  the 


prospective  ATCS  having  experience  in  IFR  operations, 
a  private  pilot’s  license,  or  ATCS  experience,  the  odds  of 
passing  Academy  training  also  increased. 

In  Table  6,  the  95%  confidence  interval  (Cl)  for  the 
odds  ratios  for  each  of  the  predictors  in  the  model  is  pre¬ 
sented.  Limits  of  the  Cl  for  all  predictors  were  greater  than 
1,  indicating  that  if  prospective  ATCSs  have  experience 
in  IFR  operations,  as  a  private  pilot,  or  as  an  ATCS,  they 
will  be  more  likely  to  pass  Academy  training  than  if  they 
have  not  had  those  experiences. 

The  remaining  items  did  not  make  a  significant  im¬ 
provement  in  predicting  Academy  training  performance 
and  were  not  included  in  the  model. 
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Table  6.  95%  Confidence  Intervals  (Cl)  for  the  Odds  Ratio 


B(SE) 

Lower 

Odds  Ratio 

Upper 

Included 

Constant 

.049  (.024) 

IFR  Operations  Experience 

.274  (.063) 

1.163 

1.315 

1.487 

Private  Pilot’s  License 

.260  (.066) 

1.140 

1.296 

1.474 

ATCS  Experience 

.298  (.112) 

1.082 

1.347 

1.677 

STUDY 2 
METHOD 

This  study  was  conducted  to  assess  the  degree  to  which 
OPM  Alternate  Requirements  6-7  (see  Table  1  for  a  com¬ 
plete  listing  of  the  OPM  alternate  requirements)  listed 
in  the  OPM  qualification  standard  for  ATC  Series  2152 
were  related  to  Academy  training  performance.  Alternate 
Requirement  6  referred  to  having  experience  as  a  rated 
Aerospace  Defense  Command  Intercept  Director,  but  no 
BQitem  could  be  identified  as  relating  directly  or  indirectly 
to  such  experience.  Similarly  for  Alternate  Requirement 
7,  which  referred  to  a  pay  schedule  that  is  no  longer  used 
for  ATCSs  and  a  written  test  no  longer  taken  by  ATCS 
applicants,  there  were  no  objective  data  available  to  assess 
this  requirement.  Therefore,  a  qualitative  approach  based 
on  interview  data  and  a  review  of  pertinent  documents 
was  used  to  assess  these  alternative  requirements. 

Participants 

Military  subject  matter  experts  (SMEs)  from  the 
U.S.  Air  Force,  U.S.  Army,  U.S.  Navy,  and  U.S.  Marine 
Corps  with  expertise  in  military  ATC  were  interviewed 
to  assess  Alternate  Requirement  6.  Specialists  in  FAA  hu¬ 
man  resources  were  also  interviewed  to  assess  Alternate 
Requirement  7. 

Procedures 

The  military  SMEs  were  asked  to  describe  the  selec¬ 
tion,  training,  placement,  and  assessment  of  military  air 
traffic  controllers.  They  were  also  asked  to  identify  any 
jobs  within  the  military  that  required  similar  knowledge, 
skills,  and  abilities  as  an  air  traffic  controller.  FAA  human 
resources  personnel  were  interviewed  and  asked  to  describe 
the  FAA  ATCS  applicant  qualification  process.  Supporting 
military  and  civilian  documentation  was  also  reviewed. 

RESULTS 

Alternate  Requirement  6 

Have  served  as  a  rated  Aerospace  Defense  Command 
Intercept  Director.  The  Aerospace  Defense  Command 
Intercept  Director  was  a  previous  job  title  within  the  U.S. 


Air  Force,  but  the  title  of  the  j  ob  was  changed  to  Aerospace 
Control  and  Warning  Systems  on  30  April  2007  and  then 
on  0 1  December  2009  to  the  current  title,  Command  and 
Control  Battle  Management  Operations.  The  Air  Force 
Specialty  Code  (AFSC)  for  this  job  is  1C5X1.  AFSCs  are 
a  combination  of  numbers  and  letters  used  to  identify  a 
job  specialty  within  the  USAF.  AFSC  1C5X1  is  a  clas¬ 
sification  code  for  an  enlisted  airman  (AFECD,  2011). 

Each  position  in  the  AFSC  represents  an  identifier 
of  the  job  with  the  combination  of  numbers  and  letters 
identifying  a  unique  job  within  the  USAF  A  breakdown 
of  the  1C5X1  AFSC  is  shown  in  Table  7.  The  first  number 
identifies  the  career  group.  The  1  in  the  first  position  is  for 
operations.  The  second  position  is  the  career  field.  The  1 C 
indicates  that  this  job  is  in  the  command  and  control  sys¬ 
tems  operations  career  field.  The  third  number  combined 
with  the  first  two  characters  of  the  AFSC  indicates  the 
career  field  subdivision.  The  fourth  character,  a  number, 
identifies  the  skill  level.  An  X  is  used  to  designate  all  skill 
levels.  The  fifth  character,  also  numeric,  combined  with 
the  other  four  numbers  and  letters,  identifies  the  specific 
AFSC.  AFSCs  may  also  include  a  prefix  or  suffix  (called 
a  Shredout)  to  identify  additional  specialty  information. 
For  example,  a  senior  airman  with  an  AFSC  of  1C5X1 
who  has  received  specialized  training  as  a  weapons  director 
has  the  D  suffix  added  to  his  or  her  specialty  code  (AFSC 
1C5X1D).  Each  USAF  job  has  both  a  title  and  an  AFSC 
(Air  Force  Instruction  36-2101,  2010,  June  14). 

Detailed  information  on  the  duties  and  responsibilities 
of  AFSC  1 C5X1  as  well  as  the  required  personnel  qualifica¬ 
tions  are  provided  in  the  Air  Force  Enlisted  Classification 
Directory  (AFECD,  2011).  The  current  AFECD  (201 1) 
is  the  official  guide  to  the  USAF  enlisted  classification 
code  and  is  primarily  used  by  USAF  personnel  officials 
and  agencies  engaged  in  procurement,  classification,  and 
training.  The  classification  system  used  in  the  AFECD  is 
detailed  in  Air  Force  Instruction  36-2101  (2010). 

The  AFSC  1C5X1  summary,  duties  and  responsibili¬ 
ties,  and  qualifications  have  been  updated  with  each  title 
change,  but  basically  the  job  has  remained  the  same.  AFSC 
1C5X1  and  1C5X1D  airmen  are  trained  to  work  with 
deployed  units  to  control  air  traffic  during  times  of  war 
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Table  7.  Air  Force  Specialty  Code  Explained 


Character 

Identifiers 

Example 

1C5X1D 

First  Position  (Numeric) 

Career  Group 

1 -Operations,  2-Fogistics,  3- 
Support,  4-Medical  or  Dental, 
5-Fegal  or  Chaplain,  6- 
Acquisition  or  Finance,  7- 
Special  Investigation,  8- 
Special  Duty  Identifier,  9- 
Reporting  Identifier 

1 

Operations 

Second  Position  Combined 
with  First  Character  (Alpha) 

Career  Field 

1C 

Command  and  Control 
Systems  Operations 

Third  Position  Combined 
with  First  and  Second 
Character  (Numeric) 

Career  Field  Subdivision 

1C5 

Command  and  Control  Battle 
Management  Operations 

Fourth 

(Numeric) 

Skill  Fevel  of  AFSC 

1 -Helper 

3 -Apprentice 

5 -Journeyman 

7-Craftsman 

9-Superintendent 

0-Chief  Enlisted  Manager 

1C5X 

X  Represents  All  Skill  Levels 

Fifth  Combined  with  Other 
Four  Characters  (Numeric) 

Specific  AFSC 

1C5X1 

Alpha  Prefix  or  Suffix 
(Shredout) 

Positions  Associated  With 
Particular  Equipment  or 
Functions  Within  a  Single 
Specialty 

1C5X1D 

Completion  of  Weapons 
Director  Training 

(J.  Kirk,  personal  communication,  August  24,  2010).  In 
their  role  as  wartime  air  traffic  controllers,  the  airmen  in 
Command  and  Control  Battle  Management  Operations 
( 1 C5X1 )  control  fighters,  tankers,  and  bombers  using  both 
radar  and  procedural  processes;  they  oversee  air  traffic  dur¬ 
ing  air-to-air  combat  and  combat  air  refueling  operations, 
and  they  keep  track  of  aircraft  during  forced  marshalling 
operations  in  which  there  are  no  pre-defined  flight  plans 
(D.  Ennis,  personal  communication,  January  18,  201 1). 

AFSC  1C5X1  is  for  enlisted  airmen.  However,  there 
are  also  officers  in  this  career  field.  Officer  positions  are 
identified  with  a  separate  AFSC  (Air  Force  Instruction 
36-2 101,2010).  These  positions  include  authorized  codes, 
title,  summary,  duties  and  responsibilities,  qualifications, 
and  shredouts  (if  applicable)  and  are  contained  in  the  Air 
Force  Officer  Classification  Directory  (AFOCD,  2011). 
The  officer  AFSC  -  associated  with  the  job  title  Command 
and  Control  Battle  Management  Operations  is  AFSC 
13BX.  Officers  in  Command  and  Control  Battle  Man¬ 
agement  Operations  perform  similar  tasks  as  the  enlisted 
personnel  in  AFSC  1C5X1,  especially  as  company-grade 


officers  (first  lieutenant,  second  lieutenant,  and  captain) . 
They  perform  these  duties  from  both  ground  and  airborne 
platforms.  However,  as  these  officers  are  promoted  to 
field  grade  (major,  lieutenant  colonel,  and  colonel),  their 
jobs  will  likely  shift  from  hands-on  control  to  supervi¬ 
sion  and  leadership  (J.  Kirk,  personal  communication, 
August,  24,  2010). 

Military  SMEs  in  ATC  and  Command  and  Control 
Battle  Management  Operations  identified  AFSC  s  1 C5X1 , 
1C5X1D,  and  13BX  as  having  similar  training  and  skill 
requirements  as  the  USAF  ATC  career  field  (1C1X1). 
Currently,  there  are  no  objective  data  to  determine  the 
relationship  between  having  experience  in  Command  and 
Control  Battle  Management  Operations  and  civilian  ATC 
performance,  but  the  similarity  of  the  jobs  suggests  that 
collecting  additional  data  using  the  updated  title  for  the 
career  field  to  assess  the  relationship  is  warranted.  There¬ 
fore,  we  recommend  that  an  item  be  added  to  the  BQ  to 
assess  the  extent  to  which  applicants  with  experience  in 
Command  and  Control  Battle  Management  Operations 
perform  well  in  Academy  training. 
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The  Aerospace  Defense  Command  Intercept  Director 
was  not  a  military  position  within  any  of  the  other  services, 
and  no  other  military  jobs  were  identified  by  the  military 
SMEs  as  having  knowledge,  skills,  and  abilities  similar  to 
those  of  an  air  traffic  controller. 

Alternate  Requirement  7 

Meet  the  requirements  for  GS-5  and  pass  the  written 
test  with  an  appropriately  higher  score.  Alternate  Require¬ 
ment  7  was  made  irrelevant  by  changes  to  both  the  FAA 
pay  scale  for  prospective  ATCSs  and  testing  requirements. 
The  General  Schedule  (GS)  pay  scale  system  is  no  longer 
used  to  classify  ATCSs.  Prospective  ATCSs  are  selected 
based  on  the  vacancy  announcement  to  which  they  apply. 
Each  vacancy  announcement  details  the  criteria  for  selec¬ 
tion,  training  requirements,  and  anticipated  compensation 
level  (R.  Mitchell,  personal  communication,  n.d.).  While 
in  training  at  the  FAA  Academy,  all  prospective  ATCSs 
are  placed  on  temporary  excepted  appointments  and  paid 
accordingly.  Once  training  is  completed,  their  salaries  are 
determined  based  on  the  current  ATC  bargaining  unit 
agreement,  which  reflects  the  previous  experience  of  the 
new  ATCS  and  the  facility  level  to  which  the  new  hires 
are  assigned. 

The  OPM  written  test  is  no  longer  used  to  qualify 
applicants  under  any  ATCS  hiring  authority.  Rather, 
some  prospective  ATCSs  are  now  required  to  take  and 
pass  the  Air  Traffic— Selection  and  Training  (AT- SAT) 
test  to  qualify  for  an  ATCS  position.  Major  sources  for 
recruiting  prospective  ATCSs  include  the  general  public, 
graduates  of  an  Air  Traffic- Collegiate  Training  Initiative 
program,  retired  military  controllers,  Department  of  De¬ 
fense  civilian  controllers,  former  FAA  controllers  seeking 
reinstatement,  and  VRA  military  controllers  (Air  Traffic 
Controller  Occupational  Handbook,  2003;  Federal  Avia¬ 
tion  Administration,  2011).  Vacancy  announcements  are 
written  to  target  these  populations  or  hiring  sources.  Of 
these  potential  hiring  sources,  applicants  from  the  general 
public  and  AT-CTI  graduates  are  required  to  take  and  pass 
the  AT- SAT  test.  VRA  and  RMC  applicants  and  former 
FAA  and  DOD  controllers  generally  do  not  take  AT- SAT 
or  any  other  test  for  selection. 

DISCUSSION 

Revisions  to  the  OPM  qualification  standard  for  the 
ATC  Series  2152  occupation  are  required  because  the  cur¬ 
rent  qualification  standard  does  not  reflect  changes  that 
have  been  made  in  FAA  practices  and  policies  or  military 
job  classifications.  There  is  also  a  need  to  assess  the  quali¬ 
fication  standard  as  a  predictor  of  ATCS  performance  in 
FAA  Academy  training:  Do  prospective  ATCSs  having 
the  experiences  covered  by  the  qualification  standard  pass 
Academy  training  at  a  higher  rate  than  those  who  have 
not  had  those  experiences?  This  report  addressed  both 
requirements  using  a  multi-method  approach. 


Alternate  Requirements  1-5  were  addressed  by  analyz¬ 
ing  biographical  data  gathered  from  prospective  ATCSs 
attending  the  FAA  Academy  from  1986-1992.  Alternate 
Requirements  6  and  7  were  assessed  based  on  interviews 
with  FAA  human  resources  specialists  and  military  subject 
matter  experts  and  a  review  of  personnel  documents.  A 
summary  of  the  results  are  presented  in  Table  8.  Recom¬ 
mendations  for  revising,  retaining,  or  eliminating  the 
OPM  qualification  standard  for  the  ATC  Series  2152  are 
presented  below,  by  alternate  requirement. 

RECOMMENDATIONS 

Alternate  Requirement  1 .  Three  items  from  the  BQwere 
selected  for  use  in  determining  the  extent  to  which  holding 
or  having  held  an  appropriate  facility  rating  and  having 
actively  controlled  air  traffic  in  civilian  or  military  ATC  ter¬ 
minals  or  centers  would  predict  performance  of  prospective 
ATCSs  attending  ATC  training  at  the  FAA  Academy.  All 
three  items  selected  as  representing  Alternate  Requirement  1 
were  found  to  be  related  to  Academy  training  performance. 
Prospective  ATCSs  who  indicated  that  they  had  or  had  held 
a  CTO  or  ATCS  rating  or  had  IFR  operations  experience 
passed  Academy  training  more  than  would  be  expected  and 
withdrew  or  failed  less  than  would  be  expected.  In  addition, 
two  of  the  items  -  having  an  ATCS  rating  and  experience 
in  IFR  operations  -  contributed  significantly  to  the  logistic 
regression  model  predicting  Academy  training  performance. 
These  results  are  supported  by  previous  research,  in  which  a 
consistent,  significant  relationship  between  ATC  experience 
and  Academy  training  performance  (e.g.,  Collins,  Nye,  & 
Manning,  1 990;  VanDeventer  et  al.,  1 983)  has  been  found. 
Based  on  these  results,  it  is  recommended  that  Alternate 
Requirement  1  be  retained  as  an  OPM  alternate  require¬ 
ment  and  qualifying  standard  for  ATCSs. 

Alternate  Requirement  2.  One  item  from  the  BQwas 
selected  for  use  in  determining  the  extent  to  which  holding 
or  having  held  a  FAA  certificate  as  a  dispatcher  for  an  air 
carrier  would  predict  Academy  training  performance.  No 
relationship  was  found,  although  the  trend  was  the  same  as 
seen  for  ATC  experiences  having  a  significant  relationship. 
It  is  likely  that  the  small  number  of  respondents  indicating 
that  they  had  this  type  of  experience  influenced  the  result. 
Fess  than  1%  of  the  9339  respondents  indicated  that  they 
had  experience  as  a  dispatcher  for  an  air  carrier  in  our 
sample  of  prospective  ATCSs  hired  from  1986-1992.  Thus 
there  was  no  basis  for  making  a  clear  recommendation  for 
retaining,  revising,  or  eliminating  Alternate  Requirement 
2.  In  a  review  of  current  biographical  data  captured  from  a 
sample  of  prospective  ATCSs,  hired  from  2003-2007,  the 
number  indicating  that  they  have  had  experience  as  a  dis¬ 
patcher  for  an  air  carrier  was  51  out  of  1,308  respondents, 
or  approximately  4%.  Given  the  trend  in  the  1986-1992 
dataset  and  the  number  of  prospective  ATCSs  in  a  more 
recent  dataset  indicating  that  they  have  had  experience  as 
a  dispatcher  for  an  air  carrier,  it  is  recommended  that  Al¬ 
ternate  Requirement  2  be  retained  as  written  in  the  OPM 
qualification  standard. 
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Table  8.  Summary  of  Results 


Alternate  Requirement 

BQ  Items 

Results 

1. 

Hold  or  have  held  an  appropriate 
facility  rating  and  have  actively 
controlled  air  traffic  in  civilian 

BQ33  -  Do  you  have  a  prior 

Control  Tower  Operator  Rating? 

(n=  9,354) 

x2=  25.91,  df=l,Sig  (p=.000) 

Effect  Size  =  1 .42 

or  military  ATC  terminals  or 
centers; 

BQ34  -  Do  you  have  a  prior  Air 
Traffic  Control  Specialist  (ATCS) 
rating?  (n=9,333) 

x2=  20.93,  df=l,Sig  (p=.000) 
Effect  Size  =1.58 

Logistic  Regression  Model  Predictor  3 

Recommendation: 

Retain 

BQ35  -  Do  you  have  prior  IFR 
operations  experience?  (n  =  9,349) 

x2=  43.58,  df=l,Sig  (p=.000) 
Effect  Size  =  1 .44 

Logistic  Regression  Model  Predictor  1 

2. 

Hold  or  have  held  an  FAA 
certificate  as  a  dispatcher  for  an 
air  carrier; 

Recommendation: 

Retain 

BQ  70  -  Indicate  whether  or  not 
you  currently,  or  have  ever 
possessed  any  of  the 
ratings/certificates/licenses : 
Dispatch-air  carrier.  (n=9,339) 

X2=1.79,  df  =  1,  NS  (p=  .181) 

3. 

Hold  or  have  held  an  instrument 
flight  rating; 

Recommendation: 

Retain 

BQ  65  -  Indicate  whether  or  not 
you  currently,  or  have  ever 
possessed  any  of  the 
ratings/certificates/licenses : 
Instrument.  (n=9,334) 

X2=  11.87,  df  =  1,  Sig  (p=.001) 
Effect  Size  =  1 .42 

4. 

Hold  or  have  held  an  FAA 
certificate  as  a  navigator  or  have 
been  fully  qualified  as  a 
Navigator/Bombardier  in  the 
Armed  Forces; 

BQ  7 1  -  Indicate  whether  or  not 
you  currently,  or  have  ever 
possessed  any  of  the 
ratings/certificates/licenses : 
Navigator/Bombardier  (n=9,340) 

X2=.174,  df  =  1,  NS  (p=  .677) 

Recommendation: 

Eliminate 

5. 

Have  350  hours  of  flight  time  as 
a  copilot  or  higher  and  hold  or 
have  held  a  private  certificate  or 
equivalent  Armed  Forces  rating; 

BQ  56  -  Indicate  whether  or  not 
you  currently,  or  have  ever 
possessed  any  of  the 
ratings/certificates/licenses :  Private 
Pilot.  (n=9,340) 

X2=  20.14,  df  =  1,  Sig  (p=.000) 
Effect  Size  =  1.33 

Logistic  Regression  Model  Predictor  2 

Recommendation: 

Revise/Retain 

BQ  57  -  Indicate  whether  or  not 
you  currently,  or  have  ever 
possessed  any  of  the 
ratings/certificates/licenses : 
Commercial  Pilot.  (n=9,339) 

x2=  7.73,  df  =  1,  Sig  (p=.005) 

Effect  Size  =1.38 

6. 

Have  served  as  a  rated 

Aerospace  Defense  Command 
Intercept  Director; 

No  BQ  items  of  relevance  were 
identified. 

Air  Force  Specialty  Code  Change  to 
Command  and  Control  Battle 
Management  Operations 

Recommendation: 

Revise/Retain 

7. 

Meet  the  requirements  for  GS-5 
and  pass  the  written  test  with  an 
appropriately  higher  score. 

Recommendation: 

Eliminate 

No  BQ  items  of  relevance  were 
identified. 

Hired  as  temporary  excepted 
appointments,  converted  to  ATCS  pay 
scale  at  first  duty  station.  OPM  written 
test  no  longer  required. 
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Alternate  Requirement  3.  Holding  or  having  held 
an  instrument  flight  rating  was  found  to  be  related  to 
Academy  training  performance  based  on  an  assessment  of 
responses  to  one  question  on  the  BQ.  More  prospective 
ATCSs  endorsing  this  BQ  item  passed  Academy  train¬ 
ing  than  would  be  expected  and  withdrew  or  failed  less 
than  would  be  expected.  Again,  this  finding  is  supported 
by  previous  research  in  which  a  significant  relationship 
between  prior  ATC  experience  and  Academy  training 
performance  was  found  consistently  (e.g.,  Collins  et  ah, 
1990;  VanDeventer  et  ah,  1983).  It  is  recommended  that 
Alternate  Requirement  3  be  retained  as  an  OPM  alternate 
requirement  and  qualifying  standard  for  ATCS. 

Alternate  Requirement  4.  No  significant  relation¬ 
ship  was  found  between  holding  or  having  held  an  FAA 
certificate  as  a  navigator  or  having  been  fully  qualified  as  a 
navigator/bombardier  in  the  Armed  Forces  and  Academy 
training  performance,  as  assessed  by  a  single  item  on  the 
BQ.  Again,  the  trend  was  the  same  as  seen  previously  in 
that  prospective  ATCSs  who  indicated  that  they  had  experi¬ 
ence  as  a  navigator/bombardier  passed  Academy  training 
more  than  would  be  expected  and  withdrew  or  failed  less 
than  would  be  expected.  The  number  of  respondents  in¬ 
dicating  that  they  had  this  experience,  however,  was  very 
low,  less  than  1%  of  9,349.  In  a  review  of  a  2003-2007 
dataset,  this  trend  continued  with  only  6  of  the  1,308 
prospective  ATCSs  (again,  less  than  1%)  indicating  that 
they  had  experience  as  a  navigator/bombardier.  This  item 
was  subsequently  dropped  from  later  versions  of  the  BQ. 
Deletion  of  the  item  was  most  likely  due  to  the  small 
number  of  respondents  endorsing  the  item  and,  because 
as  new  items  were  added  to  the  BQ,  there  was  a  need 
to  delete  other  items  to  manage  the  length  of  the  BQ 
(personal  communication  Dana  Broach,  July  5,  201 1). 

It  is  likely  that  advances  in  navigation  and  bomb 
guidance  technology  have  significantly  changed  the  job 
of  navigator  and  navigator/bombardier,  making  this  al¬ 
ternate  requirement  obsolete.  At  one  time,  the  role  of  the 
military  navigator/bombardier  was  to  guide  the  airplane 
to  a  bombing  target  and  release  the  aircrafts  bomb  load. 
However,  the  term  navigator/bombardier  is  no  longer  used 
in  military  job  classification  of  enlisted  servicemen.  The 
enlisted  military  job  title  that  seems  to  be  most  closely 
aligned  with  the  navigator/bombardier  position  is  the 
aerial  gunner  (W.  Melton,  personal  communication,  12 
July,  2011).  The  USAF  aerial  gunner  specialty  code  is  the 
1A7X1  and,  as  described  in  the  AFECD  (2011),  the  aerial 
gunner  inspects,  maintains  in-flight,  and  operates  airborne 
weapon  systems.  The  aerial  gunner  performs  aerial  gunner 
functions  based  on  the  aircraft  and  mission  type  during 
integrated  air  or  ground  operations.  Another  example  of 
a  military  aerial  gunner  is  the  U.S.  Marine  Corps  military 
occupational  specialty  (MOS)  6199,  enlisted  aircrew/ aerial 
observer  gunner  (Department  of  the  Navy,  Headquarters 
of  the  Marine  Corps,  Marine  Corps  Order  1200. 17A, 
2009) .  The  MOS  6199  crewmembers  act  as  aerial  observers/ 
gunners  for  U.S.  Marine  Corps  helicopters  and  tilt- rotor 


aircraft.  There  are  no  data  to  suggest,  however,  that  the 
skills  required  to  be  an  aerial  gunner  in  todays  military  are 
the  same  as  those  required  to  be  a  navigator/bombardier. 

Also,  regardless  of  service,  the  aerial  gunner  does  not 
act  as  a  navigator.  Job  positions  of  military  navigator  that 
seem  to  be  most  closely  related  to  the  navigator/bom¬ 
bardier  are  performed  by  military  officers.  In  the  USAF 
and  as  described  in  the  AFOCD  (2011),  navigation  is 
performed  by  the  combat  systems  officers  within  the 
operations  career  field  and  naval  flight  officers  within  the 
U.S.  Navy  and  Marine  Corps  (Department  of  the  Navy, 
2009).  For  example,  the  USAF  bomber  combat  systems 
officer  (12BX)  is  an  aircraft  crewmember  whose  duties 
include  navigating  the  aircraft  and  operating  navigation 
systems  to  accomplish  assigned  missions.  As  described  in 
the  AFOCD  (2011),  the  bomber  combat  systems  officer 
must  know  flight  theory,  air  navigation,  meteorology,  fly¬ 
ing  directives,  aircraft  operation  procedures,  and  mission 
tactics.  The  U.S.  Marine  Corps  manual  ofMOSs  describes 
the  naval  flight  officer  as  a  member  of  an  aircraft  crew 
with  the  responsibility  to  assist  in  the  employment  of  the 
aircraft’s  offensive  and  defensive  weapons  (Department  of 
the  Navy,  2009).  Both  the  combat  systems  officer  and  the 
naval  flight  officer  support  aviation  operations.  No  similar 
positions  were  found  in  the  other  services.  Again,  however, 
there  are  no  data  to  suggest  that  the  skills  required  today 
to  be  a  combat  systems  officer  or  naval  flight  officer  are  the 
same  skills  required  by  navigator/bombardiers. 

Given  the  small  number  of  prospective  ATCSs  who 
had  experience  as  a  navigator  or  navigator/bombardier  in 
two  samples  (1986-1992  and  2003-2007)  and  changes  to 
the  military  classification  of  navigator/bombardier,  it  is 
recommended  that  Alternate  Requirement  4  be  eliminated 
as  an  OPM  alternate  requirement  and  qualifying  standard 
for  ATCSs.  If  retained,  however,  it  is  recommended  that 
Alternate  Requirement  4  be  revised  to  eliminate  the  term 
bombardier.  It  is  also  recommended  that  if  Alternate  Re¬ 
quirement  4  is  revised  and  retained,  a  question  be  added 
to  the  BQ  regarding  experience  as  a  navigator.  Response 
rates  to  the  BQ  question  should  be  monitored  to  reassess 
the  relationship  between  experience  as  a  navigator  and 
Academy  training  performance. 

Alternate  Requirement  5.  Two  items  from  the  BQ 
were  used  to  assess  Alternate  Requirement  5.  The  items 
addressed  having  a  private  or  commercial  pilot’s  license 
but  did  not  specify  number  of  hours  of  flight  time.  Since 
both  items  were  significantly  related  to  Academy  training 
performance,  it  is  recommended  that  Alternate  Require¬ 
ment  5  be  revised  to  eliminate  the  need  for  350  hours 
of  flight  time  and  then  retained.  This  recommendation 
is  consistent  with  previous  findings  in  which  level  of  pi¬ 
lot  rating  or  number  of  hours  flown  were  not  related  to 
Academy  attrition  (Cobb  &  Nelson,  1974).  If  Alternate 
Requirement  5  is  retained  to  include  the  350  hours  of 
flight  time,  it  is  recommended  that  an  item  be  added  to 
the  BQ  to  reassess  the  relationship  between  number  of 
hours  of  flight  time  and  Academy  training  performance. 
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Alternate  Requirement  6.  There  were  no  objective 
data  to  assess  the  relationship  between  Academy  training 
performance  and  having  served  as  a  rated  Aerospace  De¬ 
fense  Command  Intercept  Director.  It  is  recommended, 
based  on  discussions  with  USAF  subject  matter  experts 
(D.  Ennis,  personal  communication,  August  24,  2010, 
201 1;  J.  Kirk,  personal  communication)  and  a  review  of 
job  classification  descriptions,  that  alternate  requirement 
6  be  revised  and  retained.  Further,  it  is  recommended 
that  an  item  be  added  to  the  BQ  to  assess  experience  in 
wartime  ATC  and  that  the  relationship  with  Academy 
training  performance  be  analyzed. 

Alternate  Requirement  7.  It  is  recommended  that 
Alternate  Requirement  7  be  eliminated.  Changes  in  pay 
schedules  and  testing  procedures  have  made  this  alternate 
requirement  obsolete. 

Limitations.  A  possible  limitation  of  this  research  was 
the  data  used  were  gathered  more  than  20  years  ago.  As 
mentioned  previously,  there  is  some  evidence  to  suggest 
that  prospective  ATCSs  hired  between  1986  and  1 992  are 
similar  to  those  hired  today  (Cannon  &  Broach,  2011). 
However,  there  were  also  differences  in  how  the  partici¬ 
pants  used  in  this  research  and  current  ATCS  candidates 
were  trained.  During  the  1986-1992  timeframe,  training 
was  used  as  a  screening  process  to  identify  and  eliminate 
ATCS  candidates  who  did  not  have  the  potential  to  become 
fully  certified  ATCSs  (Della  Rocco,  1998).  Assignment  to 
a  facility  was  made  only  after  the  candidates  successfully 
completed  the  screening  process.  Today,  ATCS  candidates 
are  assigned  to  a  facility  before  they  enter  the  Academy. 
Candidates  with  no  experience  in  ATC  attend  a  course  in 
the  basics,  followed  by  an  initial  course  specific  to  their 
assigned  option  of  terminal,  en-route,  or  terminal  radar 
control  (TRACON).  Experienced  candidates  and  gradu¬ 
ates  of  an  accredited  AT-CTI  program  do  not  attend  the 
basics  course,  but  rather  begin  Academy  training  with  the 
initial  course.  It  is  unclear  what  impact  the  differences  in 
training  may  have  had  on  our  results.  Additional  research 
may  be  warranted  to  validate  the  results  of  this  study. 

Additional  research  might  be  possible  over  the  next 
few  years  because  the  number  of  prospective  ATCSs  being 
hired  is  increasing  as  those  ATCSs  hired  after  the  PATCO 
strike  become  retirement  eligible.  The  FAA  hired  998  new 
controllers  in  2010  and  is  projected  to  hire  more  than 
20,000  new  controllers  between  2011  and  2020  (FAA, 
2011).  There  is  an  opportunity,  with  many  new  ATCSs 
entering  the  workforce,  to  collect  biographical  data  to  both 
reevaluate  the  recommendations  made  in  this  research,  as 
well  as  to  identify  other  predictors  of  Academy  training 
performance.  These  predictors  may  also  include  personality 
dimensions.  As  noted  by  Schroeder  et  al.  (1998),  there 
will  be  an  increased  emphasis  on  coordination  and  shared 
decision  making  in  future  ATC  systems,  which  may  result 
in  personality  dimensions  such  as  teamwork  emerging  as 
a  significant  predictor  of  ATCS  performance. 


Another  limitation  is  the  use  of  Academy  training 
performance  as  the  criterion  in  our  analyses.  Civil  Aero¬ 
space  Medical  Institute  (CAMI)  researchers  are  working 
to  develop  outcome  criteria  based  on  on-the-job  training 
and  later  job  performance.  Current  research  efforts  are 
underway  to  identify  and  collect  data  from  ATCSs  as  they 
advance  through  the  stages  of  on-the-job  training  and 
become  certified  professional  controllers.  Although  it  is 
important  to  select  ATCSs  who  can  pass  FAA  Academy 
training,  our  ultimate  aim  is  to  select  ATCSs  who  will  be 
successful  on  the  job. 

Conclusion.  The  OPM  qualification  standard  for  the 
ATC  Series  2152  was  reviewed  and  recommendations  made 
to  retain,  revise  and  retain,  or  eliminate  each  of  the  seven 
alternate  requirements.  Where  possible,  recommendations 
were  made  based  on  an  analysis  of  data  gathered  from 
students  attending  ATC  training  at  the  FAA  Academy 
between  1986  and  1992.  If  no  data  were  available,  recom¬ 
mendations  were  made  based  on  interviews  with  subject 
matter  experts  or  review  of  pertinent  documents. 

Alternate  Requirements  1-3  were  recommended  for 
retention  as  written  in  the  OPM  qualification  standard. 
Two  recommendations  were  made  for  Alternate  Require¬ 
ment  4.  First,  it  was  recommended  for  elimination,  but  if 
retained,  it  should  be  revised  to  reflect  an  accurate  military 
job  title.  Alternate  Requirements  5-6  were  recommended 
for  revision  and  retention.  Alternate  Requirement  7  was 
recommended  for  elimination. 

A  final  recommendation  is  that  a  longitudinal  study 
be  initiated  to  investigate  the  revisions  made  to  the  OPM 
qualification  standard  for  the  ATC  Series  2152  and  to 
identify  additional  biographical  and  personality  predictors 
of  successful  performance  in  ATC  training  at  the  FAA 
Academy  and  for  on-the-job  training  and  performance. 
This  research  is  needed  to  identify  both  the  most  useful 
predictors  and  the  most  relevant  criteria  to  influence 
personnel  selection  decisions.  This  recommendation  is 
also  consistent  with  recommendations  made  by  the  FAAs 
Independent  Review  Panel  (IRP)  on  air  traffic  controller 
selection,  assignment,  and  training  (Barr,  Brady,  Koleszar, 
New,  Pounds,  2011).  The  IRP,  which  was  convened  by 
the  FAA  Administrator  in  2011,  reviewed  selection,  as¬ 
signment,  and  training  of  ATCSs  and  made  a  number 
of  recommendations  for  use  as  a  roadmap  to  improve 
workforce  effectiveness.  In  their  recommendations,  the 
IRP  recognized  the  importance  of  tracking  selectees  by 
hiring  source  from  selection  through  full  qualification  as 
certified  professional  controllers.  Thus,  the  use  of  longi¬ 
tudinal  studies  to  review  and  revise  selection  processes 
will  continue  to  be  an  integral  component  of  the  FAAs 
process  for  selecting  and  qualifying  ATCS  candidates. 
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APPENDIX  A: 

CTO  Rating  •  Academy  Training  Performance  Crosstabulation 


CTO  Rating 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

3994 

367 

4361 

Expected  Count 

3920.0 

441.0 

4361.0 

%  within  Academy 

Status  F/P 

91.6% 

8.4% 

100.0% 

%  within  CTO  Rating 

47.5% 

38.8% 

46.6% 

%  of  Total 

42.7% 

3.9% 

46.6% 

Std.  Residual 

1.2 

-3.5 

Passed  Count 

4414 

579 

4993 

Expected  Count 

4488.0 

505.0 

4993.0 

%  within  Academy 

Status  F/P 

88.4% 

11.6% 

100.0% 

%  within  CTO  Rating 

52.5% 

61.2% 

53.4% 

%  of  Total 

47.2% 

6.2% 

53.4% 

Std.  Residual 

-1.1 

3.3 

Total  Count 

8408 

946 

9354 

Expected  Count 

8408.0 

946.0 

9354.0 

%  within  Academy 

Status  F/P 

89.9% 

10.1% 

100.0% 

%  within  CTO  Rating 

100.0% 

100.0% 

100.0% 

%  of  Total 

89.9% 

10.1% 

100.0% 

A1 


APPENDIX  B: 

ATCS  Rating  •  Academy  Training  Performance  Crosstabulation 


ATCS  Rating 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

4196 

159 

4355 

Expected  Count 

4149.2 

205.8 

4355.0 

%  within  Academy 

Status  F/P 

96.3% 

3.7% 

100.0% 

%  within  ATCS  Rating 

47.2% 

36.1% 

46.7% 

%  of  Total 

45.0% 

1.7% 

46.7% 

Std.  Residual 

.7 

-3.3 

Passed  Count 

4696 

282 

4978 

Expected  Count 

4742.8 

235.2 

4978.0 

%  within  Academy 

Status  F/P 

94.3% 

5.7% 

100.0% 

%  within  ATCS  Rating 

52.8% 

63.9% 

53.3% 

%  of  Total 

50.3% 

3.0% 

53.3% 

Std.  Residual 

-.7 

3.1 

Total  Count 

8892 

441 

9333 

Expected  Count 

8892.0 

441.0 

9333.0 

%  within  Academy 

Status  F/P 

95.3% 

4.7% 

100.0% 

%  within  ATCS  Rating 

100.0% 

100.0% 

100.0% 

%  of  Total 

89.9% 

10.1% 

100.0% 

B1 


APPENDIX  C: 

IFR  Operations  Experience  •  Academy  Training  Performance  Crosstabulation 


IFR  Ops 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

3743 

615 

4358 

Expected  Count 

3623.8 

734.2 

4358.0 

%  within  Academy 

Status  F/P 

85.9% 

14.1% 

100.0% 

%  within  IFR  Operations 

48.1% 

39.0% 

46.6% 

%  of  Total 

40.0% 

6.6% 

46.6% 

Std.  Residual 

2.0 

-4.4 

Passed  Count 

4031 

960 

4991 

Expected  Count 

4150.2 

840.8 

4991.0 

%  within  Academy 

Status  F/P 

80.8% 

19.2% 

100.0% 

%  within  IFR  Operations 

51.9% 

61.0% 

53.4% 

%  of  Total 

43.1% 

10.3% 

53.4% 

Std.  Residual 

-1.8 

4.1 

Total  Count 

7774 

1575 

9349 

Expected  Count 

7774.0 

1575.0 

9349.0 

%  within  Academy 

Status  F/P 

83.2% 

16.8% 

100.0% 

%  within  IFR  Operations 

100.0% 

100.0% 

100.0% 

%  of  Total 

83.2% 

16.8% 

100.0% 

Cl 


APPENDIX  D: 

Dispatch-Air  Carrier  Experience  •  Academy  Training  Performance  Crosstabulation 


Dispatch-Air 

Carrier 

Total 

No 

Yes 

Academy  Status  Withdrew  or  Failed  Count 

4330 

19 

4349 

Expected  Count 

4325.3 

23.7 

4349.0 

%  within  Academy 

Status  F/P 

99.6% 

.41% 

100.0% 

%  within  Dispatch- Air  Carrier 

46.6% 

37.3% 

46.6% 

%  of  Total 

46.4% 

.2% 

46.6% 

Std.  Residual 

.1 

-1.0 

Passed  Count 

4958 

32 

4990 

Expected  Count 

4962.7 

27.3 

4990.0 

%  within  Academy 

Status  F/P 

99.4% 

.6% 

100.0% 

%  within  Dispatch- Air  Carrier 

53.4% 

62.7% 

53.4% 

%  of  Total 

53.1% 

.3% 

53.4% 

Std.  Residual 

-.1 

.9 

Total  Count 

9288 

51 

9339 

Expected  Count 

9288.0 

51.0 

9339.0 

%  within  Academy 

Status  F/P 

99.5% 

.5% 

100.0% 

%  within  Dispatch- Air  Carrier 

100.0% 

100.0% 

100.0% 

%  of  Total 

99.5% 

.5% 

100.0% 

D1 


APPENDIX  E: 

Instrument  Rating  •  Academy  Training  Performance  Crosstabulation 


Instrument  Rating 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

4187 

159 

4346 

Expected  Count 

4152.8 

193.2 

4346.0 

%  within  Academy 

Status  F/P 

96.3% 

3.7% 

100.0% 

%  within  Instrument  Rating 

46.9% 

38.3% 

46.6% 

%  of  Total 

444.9% 

1.7% 

46.6% 

Std.  Residual 

.5 

-2.5 

Passed  Count 

4732 

256 

4988 

Expected  Count 

4766.2 

221.8 

4988.0 

%  within  Academy 

Status  F/P 

94.9% 

5.1% 

100.0% 

%  within  Instrument  Rating 

53.1% 

61.7% 

53.4% 

%  of  Total 

50.7% 

2.7% 

53.4% 

Std.  Residual 

.5 

2.3 

Total  Count 

8919 

415 

9334 

Expected  Count 

8919.0 

415.0 

9334.0 

%  within  Academy 

Status  F/P 

95.6% 

4.4% 

100.0% 

%  within  Instrument  Rating 

100.0% 

100.0% 

100.0% 

%  of  Total 

95.6% 

4.4% 

100.0% 

El 


APPENDIX  F: 

Navigator/Bombardier  Experience  •  Academy  Training  Performance  Crosstabulation 


IFR  Ops 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

4328 

25 

4358 

Expected  Count 

4326.4 

26.62 

4358.0 

%  within  Academy 

Status  F/P 

99.4% 

.6% 

100.0% 

%  within  Navigator/Bombardier 

46.6% 

43.9% 

46.6% 

%  of  Total 

46.3% 

.3% 

46.6% 

Std.  Residual 

.0 

-.3 

Passed  Count 

4955 

32 

4991 

Expected  Count 

4956.6 

30.4 

4991.0 

%  within  Academy 

Status  F/P 

99.4% 

.6% 

100.0% 

%  within  Navigator/Bombardier 

53.4% 

56.1% 

53.4% 

%  of  Total 

53.1% 

.3% 

53.4% 

Std.  Residual 

.0 

.3 

Total  Count 

9283 

57 

9349 

Expected  Count 

9283.0 

57.0 

9349.0 

%  within  Academy 

Status  F/P 

99.4% 

.6% 

100.0% 

%  within  Navigator/Bombardier 

100.0% 

100.0% 

100.0% 

%  of  Total 

99.4% 

.6% 

100.0% 

FI 


APPENDIX  G: 

Private  Pilot  •  Academy  Training  Performance  Crosstabulation 


IFR  Ops 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

3889 

462 

4351 

Expected  Count 

3818.1 

532.9 

4351.0 

%  within  Academy 

Status  F/P 

89.4% 

10.6% 

100.0% 

%  within  Private  Pilot 

47.4% 

40.4% 

46.6% 

%  of  Total 

41.6% 

4.9% 

46.6% 

Std.  Residual 

1.1 

-3.1 

Passed  Count 

4307 

682 

4989 

Expected  Count 

4377.9 

611.1 

4989.0 

%  within  Academy 

Status  F/P 

86.3% 

13.7% 

100.0% 

%  within  Private  Pilot 

52.6% 

59.6% 

53.4% 

%  of  Total 

46.1% 

7.3% 

53.4% 

Std.  Residual 

-1.1 

2.9 

Total  Count 

8196 

1144 

9340 

Expected  Count 

8196.0 

1144.0 

9340.0 

%  within  Academy 

Status  F/P 

87.8% 

12.2% 

100.0% 

%  within  Private  Pilot 

100.0% 

100.0% 

100.0% 

%  of  Total 

87.8% 

12.2% 

100.0% 

G1 


APPENDIX  H: 

Commercial  Pilot  •  Academy  Training  Performance  Crosstabulation 


IFR  Ops 

No 

Yes 

Total 

Academy  Status  Withdrew  or  Failed  Count 

4222 

129 

4351 

Expected  Count 

4197.3 

153.7 

4351.0 

%  within  Academy 

Status  F/P 

97.0% 

3.0% 

100.0% 

%  within  Commercial  Pilot 

46.9% 

39.1% 

46.6% 

%  of  Total 

45.2% 

1.4% 

46.6% 

Std.  Residual 

.4 

-2.0 

Passed  Count 

4787 

201 

4988 

Expected  Count 

4811.7 

176.3 

4988.0 

%  within  Academy 

Status  F/P 

96.0% 

4.0% 

100.0% 

%  within  Commercial  Pilot 

53.1% 

60.9% 

53.4% 

%  of  Total 

41.3% 

2.2% 

53.4% 

Std.  Residual 

-.4 

1.9 

Total  Count 

9009 

330 

9339 

Expected  Count 

9009.0 

330.0 

9339.0 

%  within  Academy 

Status  F/P 

96.5% 

3.5% 

100.0% 

%  within  Commercial  Pilot 

100.0% 

100.0% 

100.0% 

%  of  Total 

96.5% 

3.5% 

100.0% 

HI 


